System and method for data anomaly detection process in assessments

ABSTRACT

A method, computer program product, and computer system for identifying at least one attribute of a user. An attention level of the user is determined with the identified at least one attribute. The attention level of the user is analyzed. An action of the user is classified as an attention deficiency event using the analyzed attention level of the user.

RELATED CASES

This application claims the benefit of U.S. Provisional Application No.61/873,443, filed on 4 Sep. 2013, and is a continuation in part of U.S.patent application Ser. No. 13/667,425 filed on 2 Nov. 2012, entitledSystem and Method for Data Anomaly Detection Process in Assessments,which claims the benefit of U.S. Provisional Application No. 61/555,748,filed on 4 Nov. 2011, the contents of which are hereby incorporated byreference.

TECHNICAL FIELD

This disclosure relates to assessment systems and methods and, moreparticularly, to assessment cheating detection systems and methods andto recording data related to the attentions levels of people payingattention to learning materials.

BACKGROUND

Assessments (e.g., exams) are used in many parts of society to measureknowledge, skills, abilities and behaviors, e.g., in order to certifypeople for job roles and qualifications or grant licenses to work orperform tasks. For instance, educational institutions may use exams tovalidate work, knowledge, and skills to give educational qualifications.As another example, during the recruitment or promotion processes, anorganization may test how a candidate behaves under certaincircumstances to see if the candidate will fit in with the culture. Asyet another example, companies including Information Technology (IT) andother high-tech companies may issue certifications to people who can useor maintain products, or who are skilled with products. Organizationsmay use internal exams to confirm that people are competent to do jobswhere failure has a high risk (e.g., financial services and trading,operators of power stations and transport operators, etc.). Governmentagencies may provide licenses-to-work based on exam results for manyprofessional trades such as doctors, nurses, crane operators, etc., andfor licenses to drive.

Some of these exams may be delivered by paper and/or remotely bycomputer, with a candidate using, e.g., a workstation or other device toanswer questions. Part of the process of conducting an exam may be tominimize cheating. Common forms of cheating may include, for example,identity fraud (e.g., where someone other than the candidate claims tobe the candidate), use of cheating materials (e.g., having access tobooks, the internet, or other resources in a closed-book exam),prompting another person giving the right answer (e.g., someone sittingby the candidate or via telephone), and copying answers (e.g., lookingat how others taking the exams at the same time are answering questionsand using the same answers).

There are many variants of cheating and with exams where little,inadequate, or no supervision is provided to the candidate, cheating maybe a problem where society may not fully trust the integrity of thecertifications, qualifications and licenses that the exams provide.

Lower stakes assessments may also be used to check understanding aftere-learning or after other on-screen learning for instance duringregulatory compliance competency checking, where employees are requiredto undergo training to teach regulations, processes and procedures andneed to pay attention both during the learning and during theassessment.

SUMMARY OF DISCLOSURE

In one implementation, a method, performed by one or more computingdevices, comprises identifying at least one attribute of a user. Anattention level of the user is determined with the identified at leastone attribute. The attention level of the user is analyzed. An action ofthe user is classified as an attention deficiency event using theanalyzed attention level of the user.

One or more of the following features may be included. Analyzing theattention level of the user may include comparing the attention level ofthe user with a second attention level, wherein the second attentionlevel may be from at least one of the user and a second user. Analyzingthe attention level of the user may include comparing the attentionlevel of the user with a difficulty level of the action of the user.Analyzing the attention level of the user may further include comparingan amount of time spent by the user to perform the action with thedifficulty level of the action.

Analyzing the attention level of the user may include identifying theaction of the user as requiring the attention level of the user to reacha threshold attention level, and determining that the attention level ofthe user is less than the threshold attention level for the action ofthe user. The action of the user may include answering one or morequestions. An alert of the attention deficiency event may be provided toat least one of the user and a second user. The at least one attributemay include blood flow velocity. The at least one attribute may includebodily movement detection. The at least one attribute may include eyeblink detection. The at least one attribute may include gaze detection.The at least one attribute may include heartbeat rate detection. The atleast one attribute may include breathing detection. The at least oneattribute may include brain electrical activity detection. The at leastone attribute may include body posture detection. The at least oneattribute may include sweat detection. The attention level of the usermay be determined with a combination of at least two attributes of theuser. The attention deficiency event may include a lack of learning bythe user during a learning process. The attention deficiency event mayinclude cheating by the user during an assessment.

In another implementation, a computer program product resides on acomputer readable medium that has a plurality of instructions stored onit. When executed by a processor, the instructions cause the processorto perform operations comprising identifying at least one attribute of auser. An attention level of the user is determined with the identifiedat least one attribute. The attention level of the user is analyzed. Anaction of the user is classified as an attention deficiency event usingthe analyzed attention level of the user.

One or more of the following features may be included. Analyzing theattention level of the user may include comparing the attention level ofthe user with a second attention level, wherein the second attentionlevel may be from at least one of the user and a second user. Analyzingthe attention level of the user may include comparing the attentionlevel of the user with a difficulty level of the action of the user.Analyzing the attention level of the user may further include comparingan amount of time spent by the user to perform the action with thedifficulty level of the action.

Analyzing the attention level of the user may include identifying theaction of the user as requiring the attention level of the user to reacha threshold attention level, and determining that the attention level ofthe user is less than the threshold attention level for the action ofthe user. The action of the user may include answering one or morequestions. An alert of the attention deficiency event may be provided toat least one of the user and a second user. The at least one attributemay include gaze detection. The at least one attribute may includebodily movement detection. The at least one attribute may include eyeblink detection. The at least one attribute may include gaze detection.The at least one attribute may include heartbeat rate detection. The atleast one attribute may include breathing detection. The at least oneattribute may include brain electrical activity detection. The at leastone attribute may include body posture detection. The at least oneattribute may include sweat detection. The attention level of the usermay be determined with a combination of at least two attributes of theuser. The attention deficiency event may include a lack of learning bythe user during a learning process. The attention deficiency event mayinclude cheating by the user during an assessment.

In another implementation, a computing system includes a processor andmemory configured to perform operations comprising identifying at leastone attribute of a user. An attention level of the user is determinedwith the identified at least one attribute. The attention level of theuser is analyzed. An action of the user is classified as an attentiondeficiency event using the analyzed attention level of the user.

One or more of the following features may be included. Analyzing theattention level of the user may include comparing the attention level ofthe user with a second attention level, wherein the second attentionlevel may be from at least one of the user and a second user. Analyzingthe attention level of the user may include comparing the attentionlevel of the user with a difficulty level of the action of the user.Analyzing the attention level of the user may further include comparingan amount of time spent by the user to perform the action with thedifficulty level of the action.

Analyzing the attention level of the user may include identifying theaction of the user as requiring the attention level of the user to reacha threshold attention level, and determining that the attention level ofthe user is less than the threshold attention level for the action ofthe user. The action of the user may include answering one or morequestions. An alert of the attention deficiency event may be provided toat least one of the user and a second user. The at least one attributemay include gaze detection. The at least one attribute may includebodily movement detection. The at least one attribute may include eyeblink detection. The at least one attribute may include gaze detection.The at least one attribute may include heartbeat rate detection. The atleast one attribute may include breathing detection. The at least oneattribute may include brain electrical activity detection. The at leastone attribute may include body posture detection. The at least oneattribute may include sweat detection. The attention level of the usermay be determined with a combination of at least two attributes of theuser. The attention deficiency event may include a lack of learning bythe user during a learning process. The attention deficiency event mayinclude cheating by the user during an assessment.

The details of one or more implementations are set forth in theaccompanying drawings and the description below. Other features andadvantages will become apparent from the description, the drawings, andthe claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustrative diagrammatic view of a data anomaly detectionprocess coupled to a distributed computing network;

FIG. 2 is an illustrative flowchart of the data anomaly detectionprocess of FIG. 1; and

FIG. 3 is an illustrative table containing information that may be usedby the data anomaly detection process of FIG. 1.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION OF THE EMBODIMENTS System Overview

As will be appreciated by one skilled in the art, the present disclosuremay be embodied as a method, system, or computer program product.Accordingly, the present disclosure may take the form of an entirelyhardware implementation, an entirely software implementation (includingfirmware, resident software, micro-code, etc.) or an implementationcombining software and hardware aspects that may all generally bereferred to herein as a “circuit,” “module” or “system.” Furthermore,the present disclosure may take the form of a computer program producton a computer-usable storage medium having computer-usable program codeembodied in the medium.

Any suitable computer usable or computer readable medium may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. The computer-usable, orcomputer-readable, storage medium (including a storage device associatedwith a computing device or client electronic device) may be, forexample, but is not limited to, an electronic, magnetic, optical,electromagnetic, infrared, or semiconductor system, apparatus, device,or any suitable combination of the foregoing. More specific examples (anon-exhaustive list) of the computer-readable medium may include thefollowing: an electrical connection having one or more wires, a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), an optical fiber, a portable compact disc read-onlymemory (CD-ROM), an optical storage device, a media such as thosesupporting the internet or an intranet, or a magnetic storage device.Note that the computer-usable or computer-readable medium could even bea suitable medium upon which the program is stored, scanned, compiled,interpreted, or otherwise processed in a suitable manner, if necessary,and then stored in a computer memory. In the context of this document, acomputer-usable or computer-readable, storage medium may be any tangiblemedium that can contain or store a program for use by or in connectionwith the instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Thecomputer readable program code may be transmitted using any appropriatemedium, including but not limited to the internet, wireline, opticalfiber cable, RF, etc. A computer readable signal medium may be anycomputer readable medium that is not a computer readable storage mediumand that can communicate, propagate, or transport a program for use byor in connection with an instruction execution system, apparatus, ordevice.

Computer program code for carrying out operations of the presentdisclosure may be written in an object oriented programming languagesuch as Java®, Smalltalk, C++ or the like. Java and all Java-basedtrademarks and logos are trademarks or registered trademarks of Oracleand/or its affiliates. However, the computer program code for carryingout operations of the present disclosure may also be written inconventional procedural programming languages, such as the “C”programming language, PASCAL, or similar programming languages, as wellas in scripting languages such as Javascript or PERL. The program codemay execute entirely on the user's computer, partly on the user'scomputer, as a stand-alone software package, partly on the user'scomputer and partly on a remote computer or entirely on the remotecomputer or server. In the latter scenario, the remote computer may beconnected to the user's computer through a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the internet using an Internet ServiceProvider).

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof apparatus (systems), methods and computer program products accordingto various implementations of the present disclosure. It will beunderstood that each block in the flowchart and/or block diagrams, andcombinations of blocks in the flowchart and/or block diagrams, mayrepresent a module, segment, or portion of code, which comprises one ormore executable computer program instructions for implementing thespecified logical function(s)/act(s). These computer programinstructions may be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the computerprogram instructions, which may execute via the processor of thecomputer or other programmable data processing apparatus, create theability to implement one or more of the functions/acts specified in theflowchart and/or block diagram block or blocks or combinations thereof.It should be noted that, in some alternative implementations, thefunctions noted in the block(s) may occur out of the order noted in thefigures. For example, two blocks shown in succession may, in fact, beexecuted substantially concurrently, or the blocks may sometimes beexecuted in the reverse order, depending upon the functionalityinvolved.

These computer program instructions may also be stored in acomputer-readable memory that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including instruction meanswhich implement the function/act specified in the flowchart and/or blockdiagram block or blocks or combinations thereof.

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed (not necessarily in a particularorder) on the computer or other programmable apparatus to produce acomputer implemented process such that the instructions which execute onthe computer or other programmable apparatus provide steps forimplementing the functions/acts (not necessarily in a particular order)specified in the flowchart and/or block diagram block or blocks orcombinations thereof.

Referring to FIG. 1, there is shown data anomaly detection process 10that may reside on and may be executed by computer 12, which may beconnected to network 14 (e.g., the internet or a local area network).Examples of computer 12 may include but are not limited to a singleserver computer, a series of server computers, a single personalcomputer, a series of personal computers, a mini computer, a tabletcomputer, a mainframe computer, or a computing cloud. The variouscomponents of computer 12 may execute one or more operating systems,examples of which may include but are not limited to: Microsoft WindowsServer™; Novell Netware™; Redhat Linux™, Unix, Mobile phone or tabletOS, or a custom operating system, for example.

As will be discussed below in greater detail, data anomaly detectionprocess 10 may detect cheating, e.g., in an assessment. For example, atleast one attribute of a user may be identified, e.g., via monitoringdevice 64, 66, 68, 70. An attention level of the user may be determinedwith the identified at least one attribute. The attention level of theuser may be analyzed. An action of the user may be classified as anattention deficiency event using the analyzed attention level of theuser.

The instruction sets and subroutines of data anomaly detection process10, which may be stored on storage device 16 coupled to computer 12, maybe executed by one or more processors (not shown) and one or more memoryarchitectures (not shown) included within computer 12. Storage device 16may include but is not limited to: a hard disk drive; a flash drive, atape drive; an optical drive; a RAID array; a random access memory(RAM); and a read-only memory (ROM).

Network 14 may be connected to one or more secondary networks (e.g.,network 18), examples of which may include but are not limited to: alocal area network; a wide area network; or an intranet, for example.

Computer 12 may include a data store, such as a database (e.g.,relational database) (not shown) and may be located within any suitablememory location, such as storage device 16 coupled to computer 12. Insome embodiments, computer 12 may utilize a database management systemsuch as, but not limited to, “My Structured Query Language” (MySQL) inorder to provide multi-user access to one or more databases, such as theabove noted relational database. The data store may also be a customdatabase, such as, for example, a flat file database or an XML database.Any other form(s) of a data storage structure may also be used. Dataanomaly detection process 10 may be a component of the database, a standalone application that interfaces with the above noted data store and/oran applet/application that is accessed via client applications 22, 24,26, 28. The above noted data store may be, in whole or in part,distributed in a cloud computing topology. In this way, computer 12 andstorage device 16 may refer to multiple devices, which may also bedistributed throughout the network.

Data anomaly detection process 10 may be accessed via clientapplications 22, 24, 26, 28. Examples of client applications 22, 24, 26,28 may include but are not limited to an assessment application,monitoring device application, standard and/or mobile web browser, emailclient application, a customized web browser, or a custom application.The instruction sets and subroutines of client applications 22, 24, 26,28, which may be stored on storage devices 30 and/or 30 a, 32 and/or 32a, 34 and/or 34 a, 36 and/or 36 a coupled to client electronic devices38, 40, 42, 44 and/or monitoring devices 64, 66, 68, 70, may be executedby one or more processors (not shown) and one or more memoryarchitectures (not shown) incorporated into client electronic devices38, 40, 42, 44 and/or monitoring devices 64, 66, 68, 70.

Monitoring devices 64, 66, 68, 70 may include known brainwave monitoringdevices and/or known gaze monitoring devices (e.g., that may includeembedded Gaze Tracking or other attention tracking software inside testdelivery software) or other attention monitoring devices, which may beenabled (e.g., via data anomaly detection process 10 and/or clientapplications 22, 24, 26, 28) to track and record data related to auser's gaze or other attention measures, stop a test if a thresholdnumber of distractions are evident (e.g., detected), record data in atransactional test taking database, convert to a data warehouse and/ordo the analysis in the transactional database to find, e.g., dataanomalies that may indicate cheating and/or correlations between auser's gaze and question difficulty to help improve a detection model.

Storage devices 30, 30 a, 32, 32 a, 34, 34 a, 36, 36 a may include butare not limited to: hard disk drives; flash drives, tape drives; opticaldrives; RAID arrays; random access memories (RAM); and read-onlymemories (ROM). Examples of client electronic devices 38, 40, 42, 44 mayinclude, but are not limited to, personal computer 38, laptop computer40, smart phone 42, notebook computer 44, a tablet (not shown), a server(not shown), a data-enabled, cellular telephone (not shown), atelevision (not shown) with one or more processors embedded therein orcoupled thereto, and a dedicated network device (not shown).Additionally/alternatively, client electronic devices 38, 40, 42, 44 mayinclude a monitoring device (e.g., monitoring device 64, 66, 68, 70).

One or more of client applications 22, 24, 26, 28 may be configured toeffectuate some or all of the functionality of data anomaly detectionprocess 10 and/or may include at least some of data anomaly detectionprocess 10. Accordingly, data anomaly detection process 10 may be apurely server-side application, a purely client-side application, or ahybrid server-side/client-side application that is cooperativelyexecuted by one or more of client applications 22, 24, 26, 28 and dataanomaly detection process 10.

Users 46, 48, 50, 52 and/or monitoring devices 64, 66, 68, 70 may accesscomputer 12 and data anomaly detection process 10 directly throughnetwork 14 or through secondary network 18. Further, computer 12 may beconnected to network 14 through secondary network 18, as illustratedwith phantom link line 54. Data anomaly detection process 10 may includeone or more user interfaces, such as browsers and textual or graphicaluser interfaces, through which users 46, 48, 50, 52 and/or monitoringdevices 64, 66, 68, 70 may access data anomaly detection process 10.

The various client electronic devices and/or monitoring devices 64, 66,68, 70 may be directly or indirectly coupled to network 14 (or network18). For example, personal computer 38 and monitoring device 64 areshown directly coupled to network 14 via a hardwired network connection.Further, notebook computer 44 and monitoring device 70 are showndirectly coupled to network 18 via a hardwired network connection.Laptop computer 40 and monitoring device 66 are shown wirelessly coupledto network 14 via wireless communication channels 56 a and 56 brespectively established between laptop computer 40 and wireless accesspoint (i.e., WAP) 58 and between monitoring device 66 and WAP 58, whichis shown directly coupled to network 14. WAP 58 may be, for example, anIEEE 802.11a, 802.11b, 802.11g, Wi-Fi, and/or Bluetooth™ device that iscapable of establishing wireless communication channel 56 a betweenlaptop computer 40 and WAP 58 and wireless communication channel 56 bbetween monitoring device 66 and WAP 58. Additionally/alternatively, amonitoring device (e.g., monitoring device 66) may be directly (and/orwirelessly) coupled to a client electronic device (e.g., clientelectronic device 40) as illustrated with phantom link line 55. Thus,information may be communicated from a monitoring device (e.g.,monitoring device 66) to a client electronic device (e.g., clientelectronic device 40), where the information may be communicated, e.g.,to computer 12 via, e.g., a network (e.g., network 14). Smart phone 42and monitoring device 68 are shown wirelessly coupled to network 14 viawireless communication channels 60 a and 60 b respectively establishedbetween smart phone 42 and cellular network/bridge 62 and monitoringdevice 68 and cellular network/bridge 62, which is shown directlycoupled to network 14.

As is known in the art, all of the IEEE 802.11x specifications may useEthernet protocol and carrier sense multiple access with collisionavoidance (i.e., CSMA/CA) for path sharing. The various 802.11xspecifications may use phase-shift keying (i.e., PSK) modulation orcomplementary code keying (i.e., CCK) modulation, for example. As isknown in the art, Bluetooth™ is a telecommunications industryspecification that allows, e.g., mobile phones, computers, smart phones,and other devices (e.g., monitoring devices 64, 66, 68, 70) to beinterconnected using a short-range wireless connection.

Client electronic devices 38, 40, 42, 44 may each execute an operatingsystem, examples of which may include but are not limited to Android™,Apple iOS™, Microsoft Windows™, Redhat Linux™, or a custom operatingsystem.

The Data Anomaly Detection Process:

As discussed above and referring also to FIGS. 2-3, data anomalydetection process 10 may identify 200 at least one attribute of a user.An attention level of the user may be determined 202 by data anomalydetection process 10 with the identified 200 at least one attribute. Theattention level of the user may be analyzed 204 by data anomalydetection process 10. An action of the user may be classified 206 bydata anomaly detection process 10 as an attention deficiency event usingthe analyzed 204 attention level of the user.

For example, data anomaly detection process 10 may detect cheating,e.g., in an assessment. For example, attributes of a user (e.g., user48), such as but not limited to brainwaves and/or a gaze of the user,may be monitored and/or measured (e.g., via data anomaly detectionprocess 10) and/or subsequently identified 200 by data anomaly detectionprocess 10. This may be achieved, for example, using, e.g., monitoringdevice 66, which may include known brainwave and/or gaze monitoringdevices (e.g., Embed Gaze Tracking software inside test deliverysoftware).

In some implementations, it may be possible to reduce live motoringrequirements for internal exams by a significant amount, perhaps from,e.g., 100% to 10%. In some implementations, the presence of attentiondetecting process 10 may also deter candidates from cheating. Warningcandidates that their gaze and other attention measures are beingtracked is likely to deter candidates from cheating. If an attentiondata anomaly is identified, it is possible to stop the currentassessment and require later live monitoring to ensure that potentialcheaters are given a chance to prove their competence by doing the testwhile being monitored by a live person; among other possibilities. Forinstance, assuming for example purposes only that live monitoring costs$20/hour, average test length is 1 hour, and an organization delivers50,000 exams per year, live monitoring may cost in the order of$1,000,000; however, gaze monitoring may reduce the number ofparticipants that may need monitoring (e.g., 10% may still require livemonitoring), which may cost, e.g., $100,000 and so savings may be in theorder of $900,000/year, just as an example.

The brainwaves and/or the gaze and/or attention of user 48 may bemonitored, measured, and identified 200 directly from monitoring device66 and/or client electronic device 40, e.g., via data anomaly detectionprocess 10. Additionally/alternatively, the brainwaves and/or the gazeand/or attention of user 48 may at any point be recorded on one or morestorage devices (e.g., storage device 32 and/or 32 a) and/orcommunicated using any appropriate means noted throughout (e.g.,wireless communication channels 56 a and 56 b, phantom link line 55,etc.) to be recorded on one or more storage devices (e.g., storagedevices 16, 30, 30 a, 32, 32 a, 34, 34 a, 36, 36 a) and subsequentlyidentified 200 by data anomaly detection process 10. Therefore, anyparticular description of where and when the brainwaves and/or the gazeand/or attention of user 48 are monitored, measured, identified 200 (andas discussed further below, determined 202, analyzed 204, and classified206, etc.) by data anomaly detection process 10 using any particulardevice(s) in any combination should be taken as an example only and notto limit the scope of the disclosure.

An attention/distraction level (e.g., attention level 302) of user 48may be determined 202 by data anomaly detection process 10 with theidentified brainwaves and/or gaze and/or other attention measurement.For instance, according to one or more example embodiments, monitoringdevice 66 and/or client electronic device 40 may include the capabilityto monitor user characteristics of user 48, such as the gaze of user 48.For example, monitoring device 66 and/or client electronic device 40 mayinclude the capability of gaze monitoring, which may be achieved using,e.g., a video camera and software to identify the facial and eyemovements. Data anomaly detection process 10 may correlate the timelooking at the screen, and/or pupil dilation, and/or eye movements,and/or, and/or looking at specific parts of the screen and/or lookingaway from the screen and/or more detailed eye vergence measurement withregard to the screen and/or facial and/or head movements related toposition of the screen. For instance, looking (or other head/eyepositioning) at a specific area of the screen may be associated by dataanomaly detection process 10 with various degrees of active thinking,cognitive load, concentration, state of mind, etc. (i.e., attentionlevel, distraction level, etc.). And looking away (or other relatedhead, neck, face, eye positioning or movement) from the screen might bean indication of possible cheating as it could mean looking at anotherdevice or at another person or at information that is not supposed to beseen, e.g., a book or “cheat sheet”.

As another example, according to one or more example embodiments,monitoring device 66 and/or client electronic device 40 may include thecapability to monitor user characteristics of user 48, such asbrainwaves and/or heart rate. For instance, the capability of brainwavemonitoring may be achieved using, e.g., Electroencephalography (EEG)technology which may measure electronic activity (e.g., varyingvoltages) within the brain from, e.g., the scalp of user 48 and theother measures for attention detection mentioned above may be used. EEGmay show oscillations at various frequency ranges. Data anomalydetection process 10 may correlate the frequency ranges and/or specialdistributions with brain activity and also compare gaze detection withhead/facial movements and other mechanisms. These may be associated bydata anomaly detection process 10 with various degrees of activethinking, cognitive load, concentration, state of mind, etc. (i.e.,attention level). However, those skilled in the art will recognize thatother techniques may be used to identify attention level 302 of user 48using, for example, brainwave monitoring and analysis 204. For example,the raw measurements of attention may be converted, e.g., by dataanomaly detection process 10, into another form (e.g., digitalrepresentation) for subsequent analysis 204. As another example, asignature of the brainwaves and/or the gaze summarizing attention level302 and/or 304 of user 48 may be used, e.g., by data anomaly detectionprocess 10, for subsequent analysis 204. As such, the specificdescription of analyzing 204 the brainwaves and/or the gaze may includeany combination of the raw and manipulated data.

As noted above, attention level 302 and/or 304 of user 48 may beanalyzed 204 by data anomaly detection process 10 using the brainwavesand/or the gaze and of user 48. For example, brainwave and/or gazeanalysis 204 may allow data anomaly detection process 10 to determinewhether user 48 is/was cheating and/or whether there is a highlikelihood that user 48 is/was cheating. For example, answeringquestions may require user 48 to make deductions, calculations, etc., todecide the correct answer properly and fairly, which may require acertain range of attention levels (as indicated, e.g., via identified200 brainwaves and/or the gaze), and if user 48 is/was cheating, therange of attention levels may be noticeably (e.g., measurably)different. For instance, user 48 may be asked identity questions aboutuser 48 and likely may require a low attention level to answer, however,when cheating with identity fraud (e.g., where user 46 takes an exam inplace of user 48), user 46 may require a higher attention level toanswer the same questions about user 48. As another example, whencheating by user 48 knowing the questions, choices, and correct answersin advance and answering based on memory rather than knowledge, theattention level required by user 48 to answer the question may beminimal. User 48 may also either answer quickly, or if user 48 finishingquickly is a cheater detection method, user 48 may answer quickly andthen “daydream” or otherwise think without concentrated attention (i.e.,a lower attention level). As another example, cheating by user 48 beingprompted by another with the correct answer or copying answers from afellow test taker, or using a cheat sheet during a closed-book exam, mayrequire a noticeably decreased attention level and/or looking in aspecific other direction by user 48. Other cheating techniques may alsobe attempted by user 48 with varying attention levels or directions ofgaze being required in the process. In some implementations, some testtakers may be visually impaired and so the gaze detection process may beunreliable for such test takers, but these people may be identified andan accommodation potentially allowed not to use the gaze identificationportion of anomaly detection for their tests. In some implementations,some test takers may have a habit of paying attention while looking inan unexpected direction (e.g., “staring into space”), but such habitsmay be detected by analyzing the data from multiple tests/exams.

Therefore, consider the following non-limiting examples of data anomalydetection process 10 analyzing 204 attention level 302 of user 48, forexample, where user 48 is performing an action (e.g., action 300) ofanswering questions on an exam:

Analyzing 204 attention level 302 of user 48 by data anomaly detectionprocess 10 may include comparing 210 attention level 302 of user 48 witha second attention level (e.g., control attention level 304), whereincontrol attention level 304 may be from at least one of user 48 and asecond user (e.g., user 48 and/or user 46). For instance, assume forexample purposes only that user 48 and the second user are the sameuser. Further assume for example purposes only that a first exam (e.g.,a calibration exercise for user 48 to determine a control norm) waspreviously taken by user 48 where the attention levels for answeringdifferent question types (e.g., easy, medium, hard) are determined 202(e.g., as control attention level 304) and where the control attentionlevel 304 a of user 48 for the difficult question type is determined202. Further assume for example purposes only that a second exam (e.g.,the real exam) is later taken by user 48 where the attention levels foranswering the same question types of the first exam are determined 202.Further assume for example purposes only that the control attentionlevel (e.g., 304 a) of user 48 answering the hard question (e.g., action300 a) on the first exam is analyzed 204 by data anomaly detectionprocess 10 to be significantly higher than the actual attention level(e.g., 302 a) of user 48 answering the hard question on the second exam,action 300 a of user 48. The extent of the analyzed 204 difference inattention level may be indicative of a cheating event (e.g., anomaly)where unauthorized help was used by user 48 to significantly lessen theattention level needed to answer the same difficulty question type. As aresult, data anomaly detection process 10 may classify 206 action 300 aas a possible cheating event using, e.g., attention level(s) 302 aand/or 304 a of user 48. Those skilled in the art will appreciate thatany number of attention levels (and/or their associated averages) forany number of users may be used as a benchmark (e.g., threshold) for thecomparison.

As used herein, the terms attention and distraction may be usedinterchangeably where appropriate. In some implementations, analyzing204 distraction level 302 of user 48 by data anomaly detection process10 may include comparing 210 distraction level 302 of user 48 with asecond distraction level (e.g., control distraction level 304), whereincontrol distraction level 304 may be from at least one of user 48 and asecond user (e.g., user 48 and/or user 46). For instance, assume forexample purposes only that user 48 and the second user are the sameuser. Further assume for example purposes only that a first exam (e.g.,a calibration exercise for user 48 to determine a control norm) waspreviously taken by user 48 where the distraction levels for answeringdifferent question types (e.g., easy, medium, hard) are determined 202(e.g., as control distraction level 304) and where the controldistraction level 304 a of user 48 for the difficult question type isdetermined 202. Further assume for example purposes only that a secondexam (e.g., the real exam) is later taken by user 48 where thedistraction levels for answering the same question types of the firstexam are determined 202. Further assume for example purposes only thatthe control distraction level (e.g., 304 a) of user 48 answering thehard question (e.g., action 300 a) on the first exam is analyzed 204 bydata anomaly detection process 10 to be significantly lower than theactual distraction level (e.g., 302 a) of user 48 answering the hardquestion on the second exam, action 300 a of user 48. The extent of theanalyzed 204 difference in distraction level may be indicative of acheating event (e.g., anomaly) where unauthorized help was used by user48 to increase the distraction level to answer the same difficultyquestion type. As a result, data anomaly detection process 10 mayclassify 206 action 300 a as a possible cheating event using, e.g.,distraction level(s) 302 a and/or 304 a of user 48. Those skilled in theart will appreciate that any number of distraction levels (and/or theirassociated averages) for any number of users may be used as a benchmark(e.g., threshold) for the comparison.

Additionally/alternatively, if data anomaly detection process 10classifies 206, e.g., action 300 a, as an attention deficiency event,(e.g., which may include a possible cheating event and/or attentiondeficiency event), data anomaly detection process 10 may provide 208alert 312 (e.g., cheat event alert 312 a) of the cheating event to atleast one of user 48 and a second user (e.g., user 50 via clientelectronic device 42). User 50 may include, for example, a monitor (alsoknown as supervisor, proctor or invigilator) or other authorityresponsible for investigating and ensuring the validity of action 300,and need not be another test taker as illustratively shown.

Data anomaly detection process 10 may classify 206 any action (e.g., 300a) as a possible cheating event, for example, on-the-fly and/or inpost-assessment analysis. If classified 206 on-the-fly, data anomalydetection process 10 (and/or the supervisor) may stop the exam, whereuser 48 may be instructed to take the exam at another time (andsometimes in another place or in a more supervised environment).Additionally/alternatively, data anomaly detection process 10 mayprovide 208 an alert (e.g., alert 312 a) to instruct user 50 to check onuser 48 or to increase the level of monitoring in person and/or by video(e.g., immediately and/or in the future) or in some other way increasevigilance or gather further data. Additionally/alternatively, ifclassified 206 by post-assessment analysis, data anomaly detectionprocess 10 may provide 208 an alert (e.g., alert 312 a) to instruct user50 to monitor user 48 closer in the future. Additionally/alternatively,if no brainwaves and/or the gaze are detected, similar action may betaken, since, e.g., otherwise one may avoid the process by, e.g.,disabling the monitoring device.

Additionally/alternatively, those skilled in the art will recognize thatthe second user (e.g., user 46) may be different than user 48. As such,control attention level 304 a may be, for example, a comparison betweenattention level 302 a used by user 46 to perform action 300 a andattention level 304 a used by user 48 to perform action 300 a. If theattention level used to perform action 300 a differs significantlybetween users 48 and 46, this may be indicative of a cheating eventwhere unauthorized help was used by one of users 48 and 50 tosignificantly lessen the attention level needed to answer the samedifficulty question type. Additionally/alternatively, those skilled inthe art will recognize that the control attention level 304 (e.g., 304a) may be an average of multiple users performing action 300 (e.g.,action 300 a) within an acceptable attention level limit. Therefore, anyparticular description of control attention level 304 should be taken asan example only and not to limit the scope of the disclosure.

Additionally/alternatively, data anomaly detection process 10 analyzing204 attention level 302 of user 48 may include comparing 212 attentionlevel 302 of user 48 with a difficulty level (e.g., difficulty level306) of action 300 taken by user 48. For example, question difficultymay be measured as, e.g., a number or p value (e.g., as a number from 0to 1), being the chance of average candidates correctly answering action300 (so a p value of 0.7 means that 70% of candidates correctly answer).For example, an “easy” question may be a 0.9 difficulty level and a“hard” question may be a 0.2 difficulty level. Other methods ofmeasuring question difficulty (e.g., item response theory) may also beused. Those skilled in the art will appreciate that other techniques ofcalculating question difficulty may also be used without departing fromthe scope of the disclosure, and that question difficulty may vary withthe test-taker's competence so that a question that is difficult for abeginner may be easy for an expert (e.g., data anomaly detection process10 may take into account what is the expected cognitive load vs. theactual cognitive load). Generally, harder questions may take longer toanswer than easier questions, and harder questions may also require ahigher attention level and/or a reduced distraction level than easierquestions. As such, an analysis 204 which compares 212 a user'sattention level while performing action 300 (e.g., answering a question)with the difficulty level (e.g., difficulty level 306) may be used bydata anomaly detection process 10 to distinguish and to classify 206 auser (e.g., user 48) as cheating and/or having a high probability ofcheating. For example, assume for example purposes only that thedifficulty level (e.g., difficulty level 306 b) of action 300 b is high(e.g., 0.2). As such, control attention level 304 b of user 48 may onaverage be high (e.g., around 8 using a scale of 1-10). Further assumefor example purposes only that user 48 performs action 300 b and dataanomaly detection process 10 determines 202 the attention level(attention level 302 b) of user 48 for performing action 300 b is (e.g.,3). The extent of the analyzed 204 difference in attention level 302 bverses difficulty level 306 b (and/or control attention level 304 b) maybe indicative of a cheating event where unauthorized help was used byuser 48 to significantly lessen the attention level needed to answer adifficult question (i.e., action 300 b). As a result, data anomalydetection process 10 may classify 206 action 300 b as a possiblecheating event using, e.g., attention level 302 b of user 48.

Additionally/alternatively, as noted above, if data anomaly detectionprocess 10 classifies 206 action 300 b as a possible cheating event,data anomaly detection process 10 may provide 208 alert 312 (e.g., cheatevent alert 312 b) of the cheating event to at least one of user 48 anda second user (e.g., user 50 via client electronic device 42).

Additionally/alternatively, data anomaly detection process 10 analyzing204 attention level 302 of user 48 may further include comparing 214 anamount of time (e.g., actual time 308 and/or control time 310) spent byuser 48 to perform action 300 with difficulty level 306 of action 300.For example, data anomaly detection process 10 may more accuratelyanalyze 204 attention level 302 of user 48 if, for example, time (e.g.,actual time 308) is also incorporated into the analysis 204. Forinstance, assume for example purposes only that the difficulty level(e.g., difficulty level 306 c) of action 300 c is high (e.g., 0.2). Assuch, control attention level 304 c of user 48 may on average be high(e.g., around 8). Further assume for example purposes only that user 48performs action 300 c and data anomaly detection process 10 determines202 the attention level (attention level 302 c) of user 48 forperforming action 300 c is (e.g., 3). Further assume for examplepurposes only that user 48 performs action 300 c in an amount of time308 c (e.g., 1 minute) that is similar to an amount of time as would beperformed with an easier action. The extent of the analyzed 204difference in attention level 302 c versus difficulty level 306 c(and/or control attention level 304 c) when also compared 214 withamount of time 308 c used by user 48 when answering action 300 c may bea stronger indication of a cheating event where unauthorized help wasused by user 48 to answer a difficult question (i.e., action 300 c). Asa result, data anomaly detection process 10 may classify 206 action 300c as a possible cheating event using attention level 302 c of user 48.

Additionally/alternatively, as noted above, if data anomaly detectionprocess 10 classifies 206 action 300 c as a cheating event, data anomalydetection process 10 may provide 208 alert 312 (e.g., cheat event alert312 c) of the cheating event to at least one of user 48 and a seconduser (e.g., user 50 via client electronic device 42).

Additionally/alternatively, further assume for example purposes onlythat user 48 performs action 300 c in an amount of time 308 c (e.g., 1minute) where a control amount of time (e.g., control time 310 c) aroundwhich user 48 should spend on average on action 300 c is, e.g., 9minutes. The extent of the analyzed 204 difference in attention level302 c, difficulty level 306 c (and/or control attention level 304 c),and amount of time 308 c used by user 48 when answering action 300 ccompared 214 with control time 310 c may be a stronger indication of acheating event where unauthorized help was used by user 48 to answer adifficult question (i.e., action 300 c). As a result, data anomalydetection process 10 may classify 206 action 300 c as a possiblecheating event using, e.g., attention level 302 c of user 48.

Additionally/alternatively, as noted above, if data anomaly detectionprocess 10 classifies 206 action 300 c as a cheating event, data anomalydetection process 10 may provide 208 alert 312 (e.g., cheat event alert312 c) of the cheating event to at least one of user 48 and a seconduser (e.g., user 50 via client electronic device 42).

Additionally/alternatively, data anomaly detection process 10 analyzing204 attention level 302 of user 48 may include identifying 216 action300 of user 48 as requiring attention level 302 of user 48 to reach athreshold attention level (e.g., control attention level 304), anddetermining 218 whether attention level 302 of user 48 is less thancontrol attention level 304 for action 300 of user 48. For example, oneor more actions may require all users to spend some time and attention.For instance, an action that only requires a user to know a fact may bedifficult, but may not necessarily require prolonged attention. Considerfor example purposes only the following two actions: (1) “What is thevalue of it to two decimal places?” (2) “There are 4 circles, one withradius 1 cm, one with radius 4 cm, one with radius 6 cm and one withradius 10 cm. What is the combined area of all four circles to twodecimal places?”

Even if a user (e.g., user 48) were a strong mathematician, user 48 maystill need to concentrate and pay attention to the second action (e.g.,action 300 d) to a certain threshold attention level (e.g., controlattention level 304 d), even if action 300 d is not considered to bethat of a high difficulty level 306 d (e.g., 0.2). Thus, it may beuseful for data anomaly detection process 10 to analyze 204 and identify216 questions which may require a higher attention level and determine218 whether attention level 302 d of user 48 is less than controlattention level 304 d for action 300 d. If data anomaly detectionprocess 10 determines 218 that attention level 302 d of user 48 is lessthan control attention level 304 d for action 300 d, data anomalydetection process 10 may classify 206 action 300 d as a cheating eventusing, e.g., attention level 302 d of user 48.

Additionally/alternatively, as noted above, if data anomaly detectionprocess 10 classifies 206 action 300 d as a cheating event, data anomalydetection process 10 may provide 208 alert 312 (e.g., cheat event alert312 d) of the cheating event to at least one of user 48 and a seconduser (e.g., user 50 via client electronic device 42).

While action 300 taken by user 48 may include the answering of one ormore questions, those skilled in the art will appreciate that othertasks may be associated with action 300. For example, action 300 mayinclude but is not limited to, performing a manual task, such as solvinga puzzle, writing a document (e.g., to detect plagiarism), reading,researching, deciding which input device (e.g., keyboard) key to press,or performing any other cognitive task. As such, any particulardescription of action 300 being a question to answer should be taken asan example only and not to limit the scope of the disclosure.

While one or more embodiments of the disclosure is described in terms ofdata anomaly detection process 10 analyzing 204 brainwaves and/or thegaze and/or other attention detection measures, those skilled in the artwill appreciate that other biological characteristics may also be usedwithout departing from the scope of the disclosure. For example,monitoring device 66 may include heart rate monitoring capabilitiesused, e.g., by data anomaly detection process 10, to determine 202 aheart rate of user 48, where the heart rate may be analyzed 204 by dataanomaly detection process 10, e.g., similarly to the operation of a liedetector. As such any particular description of analyzing brainwavesand/or the gaze and/or the attention should be taken as an example onlyand not to limit the scope of the disclosure.

Those skilled in the art will recognize that analysis 204 by dataanomaly detection process 10 may include any combination of attentionlevel 302, control attention level 304, difficulty level 306, actualtime 308, control time 310, as well as other data. Therefore, anyparticularly described combination of analyzing 204 attention level 302,control attention level 304, difficulty level 306, actual time 308, andcontrol time 310 should be taken as an example only and not to limit thescope of the disclosure. Further, while a table is shown in FIG. 3, thisis to help in the explanation of the disclosure, and those skilled inthe art will appreciate that any data format or technique may be used tomaintain the information of FIG. 3. As such, the specific format of FIG.3 should be taken as an example only and not to limit the scope of thedisclosure.

Additionally/alternatively, to aid in preventing user 48 from trickingdata anomaly detection process 10 (e.g., by using pre-recordedbrainwaves and/or the pre-recorded gaze movements and/or other attentiondetection pre-recordings or by tampering or otherwise disabling themonitoring device), certain protections may be used by data anomalydetection process 10. Such protections may include but are not limitedto, a time system within monitoring device 66, client electronic device40 and/or other device cooperatively ensuring proper use and function ofall components, and video recording user 48 using monitoring device 66,where monitoring device 66 or other device may include an appropriateserial number shown on the video or other recording.

Additionally/alternatively, data anomaly detection process 10 mayprovide feedback to user 48 at the end of performing an action (e.g.,informing user 48 which actions are right and wrong and/or givingexplanatory feedback when the action is wrong). Notably, a problem mayexist in giving such feedback, in that it may overload user 48 withdetails such that user 48 loses attention. As such, data anomalydetection process 10 may use the same or similar techniques discussedthroughout to analyze 204 attention to feedback, and then toincrease/decrease the amount of feedback depending on the attentionlevel.

Additionally/alternatively, there may be a mental difference between,e.g., retrieving information from memory and getting information fromsome other location. For instance, a different mental signature may beproduced when user 48 retrieves information from memory to answer aquestion, as opposed to when user 48 looks up the information in a book.As such, data anomaly detection process 10 may use the same or similartechniques discussed throughout to analyze 204 whether user 48 isretrieving information from memory to answer a question, or retrievingthe information from elsewhere. As a result, data anomaly detectionprocess 10 may distinguish between genuine answering of a question frommemory and being told the answer after having looked up the answerelsewhere. Therefore, data anomaly detection process 10 may analyze 204the attention of user 48 to be sure that the exam is really testing whatis in the brain of user 48, not just what user 48 answers.

Those skilled in the art will recognize that while a single user actionmay analyzed to classify whether a cheating event has occurred, thecheating event may be determined by multiple user actions withoutdeparting from the scope of the disclosure. For example, the mind ofuser 48 may wander occasionally and may or may not be a sufficientindicator of a cheating event. As such, those skilled in the art willappreciate that one or more actions may be analyzed by data anomalydetection process 10 to determine whether a threshold of actions may beclassified as a cheating event (e.g., before classifying a cheatingevent and/or providing an alert of the cheating event). The thresholdmay be adjusted, e.g., via a user interface in combination with dataanomaly detection process 10, to aid in lessening the risks of a falsepositive of classifying a cheating event. For example, if three actionsare the threshold amount of classified cheating events, then the firsttwo actions that may be normally classified as a cheating eventseparately as described above may not yet be classified as a cheatingevent, but may merely be classified as an anomaly by data anomalydetection process 10; however, if three or more actions that may beclassified as a cheating event on their own occur, then one or more ofthe three or more actions may be classified as a cheating event and notan anomaly. As such, the classification of an action (i.e., a singleaction) as a cheating event may, but need not, also include theclassification of one or more actions as anomalies for at least aportion of an assessment before data anomaly detection process 10 hasdetermined that a threshold of cheating events (or anomalies) have beenclassified. In the example, data anomaly detection process 10 may waituntil the threshold has been reached before providing an alert as notedabove. Therefore, the classification of an action (i.e., a singleaction) as a cheating event should be taken as an example only and notto otherwise limit the scope of the disclosure.

In some implementations, data anomaly detection process 10 may be usedfor lower stakes quizzes and tests, e.g., where if the person taking thequiz or test is not paying attention, the consequences may be milder,e.g., reminding the person to pay attention. In some implementations,harvesting the evidence that the person was paying attention during thelearning and assessment may be useful evidence in regulatory compliancethat the person has been through training, since there may be arequirement on companies to prove and document that compliance traininghas taken place.

While one or more implementations of the present disclosure may bedescribed as classifying 206 an action of the user as a cheating event(e.g., where the attention deficiency event may include a cheating eventby the user during an assessment) using the analyzed attention level ofthe user, it will be appreciated that the action of the user may beclassified 206 as events other than a cheating event without departingfrom the scope of the present disclosure. For example, in someimplementations, as may be equally applicable for the presentdisclosure, the action of the user may be classified 206 as an attentiondeficiency event using the analyzed attention level of the user. Forinstance, in some implementations, the attention deficiency event mayinclude a lack of learning by the user during a learning process. Forexample, for on screen learning (e.g., e-learning) where learningmaterials (for example a presentation, a video, some text to read, etc.)is followed by an assessment. In the example, determining 202 and/oranalyzing 204 the attention level of the user during the learningprocess may be useful evidence that the person had followed the learningand may supplement the information from the assessment about theirunderstanding. As such, the description of classifying 206 an action ofthe user as a cheating event using the analyzed attention level of theuser should be taken as an example only and not to limit the scope ofthe disclosure. Similarly, the description of classifying 206 an actionof the user as any type of attention deficiency event may be equallyapplicable for the present disclosure with tasks other than assessments,such as the above-noted learning process. For instance, classifying 206that the user is not paying attention during the learning process maycause data anomaly detection process 10 to provide 208 an alert of theattention deficiency event to at least one of the user and a seconduser, which may result in a change in action, e.g., halting thelearning, reminding the user to pay attention, etc. As such, thedescription of using assessments should be taken as an example only andnot to limit the scope of the disclosure.

While the present disclosure may be described using the attribute(s) ofgaze and/or brainwaves of user 48, it will be appreciated that otherexamples of attributes of user 48 may be used (e.g., identified 200) bydata anomaly detection process 10 to determine 202 and/or analyze 204the attention level of user 48 without departing from the scope of thedisclosure. For example, the at least one attribute may include bodilymovement detection. Bodily movement detection may include, e.g.,measurement of head, face, jaw, tongue and neck muscle and skeletalmovement that may indicate attention, including the 3D movement (e.g.,pitch, yaw, and roll) of different parts of the head, including the useof facial muscle activity (e.g., measured by facial electromyography),including measuring changes in the contraction of the corrugator muscleand zygomatic muscle. Similar to gaze detection, it may be possible thatif the eyes of user 48 move to look at something, then other parts ofthe face or head and related areas may also move, which when identified200 may help data anomaly detection process 10 to determine 202 and/oranalyze 204 the attention level of user 48.

As another example, the at least one attribute may include eye blinkdetection. The eye blink detection may include, e.g., identifying 200the duration and frequency of blinks. For instance, assume there may bea correlation between the amounts of blinking and the impact onattention. For example, when user 48 loses attention, user 48 may reduceeye movement and also do less blinking, which when identified 200 mayhelp data anomaly detection process 10 to determine 202 and/or analyze204 the attention level of user 48.

As another example, the at least one attribute may include heartbeatrate detection. The heartbeat rate detection may include, e.g.,heartbeat rate increase, decrease, rate of acceleration, rate ofdeceleration, etc. via, e.g., direct measurement and/or via pulsepoints. For instance, assume there may be a correlation between theheart rate and the impact on attention. For example, when user 48 losesattention, user 48 may have an increased and/or reduced heart rate,which when identified 200 may help data anomaly detection process 10 todetermine 202 and/or analyze 204 the attention level of user 48.

As another example, the at least one attribute may include blood flowvelocity. The blood flow velocity may include, e.g., blood flow velocityin the brain (or other locations in the body). In some implementations,blood flow velocity may be measured with, e.g., transcranial Dopplersonography, as well as other devices and/or techniques. For example,when user 48 loses attention, user 48 may have an increased and/orreduced blood flow velocity, which when identified 200 may help dataanomaly detection process 10 to determine 202 and/or analyze 204 theattention level of user 48.

As another example, the at least one attribute may include breathingdetection. The breathing detection may include, e.g., breathingfrequency, depth of breathing, nature of breathing, etc. For instance,assume there may be a correlation between the characteristics of thebreathing and the impact on attention. For example, when user 48 losesattention, user 48 may have an increased and/or reduced breathing rate,which when identified 200 may help data anomaly detection process 10 todetermine 202 and/or analyze 204 the attention level of user 48.

As another example, the at least one attribute may include brainelectrical activity detection. The brain electrical activity mayinclude, e.g., brain electrical activity from the frontal, temple,parietal and/or perceptual areas of the brain, as well as other brainelectrical activity. For instance, assume there may be a correlationbetween the characteristics of the brain electrical activity and theimpact on attention. For example, when user 48 loses attention, user 48may have an increased and/or reduced brain electrical activity, whichwhen identified 200 may help data anomaly detection process 10 todetermine 202 and/or analyze 204 the attention level of user 48.

As another example, the at least one attribute may include body posturedetection. Body posture may include, e.g., body posture such asslouching, turning to one's side, moving one's head to look away fromwhere a question may be located, etc. (e.g., measured by ultrasound,sonar, echo location, visual means or otherwise) and movements of otherparts of body, e.g., hands or feet. For instance, user 48 may be lookingat another user or at some other location that may have the rightanswers, which when identified 200 may help data anomaly detectionprocess 10 to determine 202 and/or analyze 204 the attention level ofuser 48.

As yet another example, the at least one attribute may include sweatdetection. Sweat detection may include measurements of skin conductanceand/or electrodermal activity. For example, when user 48 losesattention, user 48 may have an increased and/or reduced skin conductance(which may also measure physiological arousal that may be a furtherprediction of attention), which when identified 200 may help dataanomaly detection process 10 to determine 202 and/or analyze 204 theattention level of user 48. As such, the description of using theattributes of gaze and/or brainwaves of user 48 (as well as any otherexamples of attributes) should be taken as an example only and not tolimit the scope of the disclosure.

In some implementations, more than one monitoring device may be used forthe monitoring, with the results correlated together using an attentionmeasurement function. For instance, assume for example purposes onlythat two monitoring devices are being used by data anomaly detectionprocess 10. In the example, if the first device measure indicatesattention and the second device measure indicates inattention (asdiscussed above), the conflicting classification 206 of an attentionlevel deficiency event may be less strong an indication of a finalclassification 206 of an attention level deficiency event than if bothindicated inattention. Conversely, if the two or more measures correlate(i.e., they both indicate the same or similar classifications 206), thecorrelating classification 206 of an attention level deficiency eventmay be stronger an indication of a final classification 206 of anattention level deficiency event.

In some implementations, the attention level of the user may bedetermined 202 with a combination of at least two attributes of the user(e.g., identified 200 using one or more of the above-noted monitoringdevices). For example, data anomaly detection process 10 may identify200 singularly and/or in any combination of attributes of the user(e.g., such as the above-noted attributes), determine 202 an attentionlevel of the user with the identified 200 attributes, analyze 204 theattention level of the user based upon, at least in part, the determined202 attention level of the user, and classify 206 an action of the useras an attention level deficiency event using the analyzed 204 attentionlevel of the user. For instance, the nature of the attention levelmeasurement is such that a series of different variables that mayinclude a plurality of the above-noted variables may be analyzed 204singularly and/or in any combination, and may be based on, at least inpart, previous sampling of the user, and may then produce a measure ofattention combined from a combination of variables. As such, thedescription of identifying 200 only a single attribute should be takenas an example only and not to limit the scope of the disclosure.

The terminology used herein is for the purpose of describing particularimplementations only and is not intended to be limiting of thedisclosure. As used herein, the singular forms “a”, “an” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will be further understood that the terms“comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, integers, steps (notnecessarily in a particular order), operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, integers, steps (not necessarily in a particular order),operations, elements, components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present disclosure has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the disclosure in the form disclosed. Many modifications,variations, and any combinations thereof will be apparent to those ofordinary skill in the art without departing from the scope and spirit ofthe disclosure. The implementation(s) were chosen and described in orderto best explain the principles of the disclosure and the practicalapplication, and to enable others of ordinary skill in the art tounderstand the disclosure for various implementation(s) with variousmodifications and/or any combinations of implementation(s) as are suitedto the particular use contemplated.

Having thus described the disclosure of the present application indetail and by reference to implementation(s) thereof, it will beapparent that modifications, variations, and any combinations ofimplementation(s) (including any modifications, variations, andcombinations thereof) are possible without departing from the scope ofthe disclosure defined in the appended claims.

What is claimed is:
 1. A computer-implemented method comprising:identifying, by at least one computing device of one or more computingdevices, at least one attribute of a user; determining, by at least onecomputing device of the one or more computing devices, an attentionlevel of the user with the identified at least one attribute; analyzing,by at least one computing device of the one or more computing devices,the attention level of the user; and classifying, by at least onecomputing device of the one or more computing devices, an action of theuser as an attention deficiency event using the analyzed attention levelof the user, wherein the attention deficiency event includes anindication of possible cheating by the user during an assessment.
 2. Thecomputer-implemented method of claim 1 wherein analyzing the attentionlevel of the user includes comparing the attention level of the userwith a second attention level, wherein the second attention level isfrom at least one of the user and a second user.
 3. Thecomputer-implemented method of claim 1 wherein analyzing the attentionlevel of the user includes comparing the attention level of the userwith a difficulty level of the action of the user.
 4. Thecomputer-implemented method of claim 3 wherein analyzing the attentionlevel of the user further includes comparing an amount of time spent bythe user to perform the action with the difficulty level of the action.5. The computer-implemented method of claim 1 wherein analyzing theattention level of the user includes: identifying the action of the useras requiring the attention level of the user to reach a thresholdattention level; and determining that the attention level of the user isless than the threshold attention level for the action of the user. 6.The computer-implemented method of claim 1 wherein the action of theuser includes answering one or more questions.
 7. Thecomputer-implemented method of claim 1 further comprising providing analert of the attention deficiency event to at least one of the user anda second user.
 8. The computer-implemented method of claim 1 wherein theat least one attribute includes gaze detection.
 9. Thecomputer-implemented method of claim 1 wherein the at least oneattribute includes bodily movement detection.
 10. Thecomputer-implemented method of claim 1 wherein the at least oneattribute includes eye blink detection.
 11. The computer-implementedmethod of claim 1 wherein the at least one attribute includes blood flowvelocity.
 12. The computer-implemented method of claim 1 wherein the atleast one attribute includes heartbeat rate detection.
 13. Thecomputer-implemented method of claim 1 wherein the at least oneattribute includes breathing detection.
 14. The computer-implementedmethod of claim 1 wherein the at least one attribute includes brainelectrical activity detection.
 15. The computer-implemented method ofclaim 1 wherein the at least one attribute includes body posturedetection.
 16. The computer-implemented method of claim 1 wherein the atleast one attribute includes sweat detection.
 17. Thecomputer-implemented method of claim 1 wherein the attention level ofthe user is determined with a combination of at least two attributes ofthe user.
 18. A computer program product residing on a non-transitorycomputer readable medium having a plurality of instructions storedthereon which, when executed by a processor, cause the processor toperform operations comprising: identifying at least one attribute of auser; determining an attention level of the user with the identified atleast one attribute; analyzing the attention level of the user; andclassifying an action of the user as an attention deficiency event usingthe analyzed attention level of the user, wherein the attentiondeficiency event includes an indication of possible cheating by the userduring an assessment.
 19. The computer program product of claim 18wherein the at least one attribute includes at least one of a gazedetection, a bodily movement detection, an eye blink detection, a bloodflow velocity, a heartbeat rate detection, a breathing detection, abrain electrical activity detection, a body posture detection, and asweat detection.
 20. The computer program product of claim 18 whereinthe attention level of the user is determined with a combination of atleast two attributes of the user.
 21. The computer program product ofclaim 18 wherein analyzing the attention level of the user includes:identifying the action of the user as requiring the attention level ofthe user to reach a threshold attention level; and determining that theattention level of the user is less than the threshold attention levelfor the action of the user.
 22. A computing system including a processorand memory configured to perform operations comprising: identifying atleast one attribute of a user; determining an attention level of theuser with the identified at least one attribute; analyzing the attentionlevel of the user; and classifying an action of the user as an attentiondeficiency event using the analyzed attention level of the user, whereinthe attention deficiency event includes an indication of possiblecheating by the user during an assessment.
 23. The computing system ofclaim 22 wherein the at least one attribute includes at least one of agaze detection, a bodily movement detection, an eye blink detection, ablood flow velocity, a heartbeat rate detection, a breathing detection,a brain electrical activity detection, a body posture detection, and asweat detection.
 24. The computing system of claim 22 wherein theattention level of the user is determined with a combination of at leasttwo attributes of the user.
 25. The computing system of claim 22 whereinanalyzing the attention level of the user includes: identifying theaction of the user as requiring the attention level of the user to reacha threshold attention level; and determining that the attention level ofthe user is less than the threshold attention level for the action ofthe user.